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Abstract 

A  program  initiated  by  the  Defense  Advanced  Research  Project  Agency  (DARPA)  called 
“Learning  Applied  to  Ground  Robots”  (LAGR)  is  developing  control  algorithms  that  would 
allow  a  vehicle  to  safely  travel  cross-country.  The  University  of  Idaho  with  funding  from  the 
Office  of  Naval  Research  is  participating  in  this  program  in  which  DARPA  provided  a  vehicle 
having  both  sensors  and  supporting  software  for  the  sensors.  We  used  the  LAGR  vehicle  to  help 
solve  navigation  problems  that  afflicts  both  underwater  crawlers  and  ground  vehicles.  With  the 
application  of  fuzzy  logic  and  specialized  software,  we  were  able  to  successfully  autonomously 
navigate  in  an  unstructured  environment  to  a  specific  target  or  location. 


INTRODUCTION 


The  surf  zone  is  a  very  challenging  environment  for  underwater  vehicles  conducting  mine- 
counter-measures  (MCM).  A  number  of  different  vehicles  and  methods  can  be  used  to  identify 
objects  that  may  be  potential  mines,  including  small  submarines  swimming  outside  the  surf  zone, 
as  well  as  aerial  and  surface  vehicles.  Once  an  object  has  been  identified  as  a  potential  mine,  a 
small  underwater  two-track  vehicle,  called  a  crawler,  can  be  used  to  reacquire  and  confirm 
identification,  and  target  the  mine.  This  mission  requires  the  crawler  to  navigate  through 
difficult  terrain  to  the  potential  mine  object  and  is  similar  to  the  problem  of  navigating  cross¬ 
country  in  an  outdoor  environment. 

DARPA  is  funding  a  program  called  “Learning  Applied  to  Ground  Robots”  (LAGR)  for  the 
purpose  of  developing  control  algorithms  that  would  allow  a  vehicle  to  safely  travel  cross¬ 
country.  The  University  of  Idaho  has  been  selected  to  participate  in  this  program  in  which 
DARPA  would  supply  a  vehicle  having  both  sensors  and  supporting  software  for  the  sensors. 
Because  this  technology  has  applications  to  both  ground  and  underwater  vehicles,  we  proposed 
that  ONR  support  one  graduate  student  and  the  maintenance  costs  for  the  DARPA  vehicle  (i.e. 
$15K)  in  order  to  integrate  methods  developed  at  the  UI  with  DARPA’ s  sensors  and  software  to 
help  solve  this  navigation  problem  that  afflicts  both  underwater  crawlers  and  ground  vehicles. 

Researchers  at  the  University  of  Idaho  (UI)  proposed  to  develop  a  fuzzy  logic  control  system  to 
allow  a  vehicle  to  autonomously  travel  cross-country  in  an  outdoor  or  underwater  environment. 
This  work  was  based  upon  a  prototype  fuzzy  logic  control  system  for  an  autonomous  vehicle 
used  in  timber  harvesting  applications  that  follows  an  ill-defined  trail  in  an  outdoor  environment 
with  a  limited  set  of  sensors,  see  previous  research  vehicles  in  Figure  1.  In  the  proposed  work, 
we  developed  a  hierarchical  fuzzy  logic  control  system  where  lower  level  modules  will  process 
data  and  provide  recommendations  to  a  supervisory  control  module.  A  critical  element  of  the 
new  control  system  is  the  Trail  Selection  Module.  This  module  identifies  potential  trail 
segments  for  the  vehicle  in  both  the  near  and  far  fields  and  defines  a  trail  for  the  vehicle.  The 
system  uses  visual  cues  to  help  identify  where  potential  trail  segments  are  located  and  attempt  to 
assemble  these  segments  so  the  vehicle  can  move  toward  a  location  in  the  far  field. 

Computer  simulations  of  the  robot  and  its  environment,  and  an  optimization  process  are  used  to 
individually  train  or  teach  each  module.  We  adapted  an  ONR  developed  software  package 
(ALSWE)  to  simulate  the  outdoor  environment  as  learned  during  monthly  and  semi-annual  runs 
performed  at  DARPA’ s  facilities.  Each  module  was  optimized  individually  so  that  if  one  or 
more  modules  should  fail,  the  Supervisory  Module  in  concert  with  the  remaining  modules  would 
operate  in  some  optimal  manner.  The  weighting  of  control  recommendations  from  lower  level 
modules  is  determined  by  a  quality  factor.  These  quality  factors  allow  the  supervisory  control 
system  to  distinguish  and  prioritize  inputs  and  thus  make  decisions  based  upon  competing  inputs 
from  the  lower  level  controllers.  The  quality  factors  are  also  viewed  as  fuzzy  quantities  and  the 
control  system  uses  them  to  “think”  about  what  recommendations  are  better  than  others  and  how 
to  use  these  recommendations. 


Defining  a  quantitative  index  for  optimizing  a  module’s  performance  is  sometimes  difficult.  For 
instance,  a  vision  system  provides  a  large  amount  of  information  that  can  be  easily  interpreted  by  a 
human  but  cannot  be  easily  interpreted  by  a  control  system.  Correctly  assessing  the  quality  factor  of  a 
module  in  a  quantitative  manner  can  also  be  difficult  although,  again,  an  expert  operator  can  many 
times  perform  the  assessment  easily.  By  utilizing  an  innovative  technique  we  developed,  called  human 
embedding,  an  expert  can  help  generate  the  metric  for  training  a  fuzzy  logic  system.  The  performance 
index  is  a  metric  between  the  expert’s  responses  and  the  fuzzy  logic  module’s  responses.  For  this 
procedure  to  converge,  the  correct  quantities  must  be  observed  and  the  correct  logic  used.  We  used 
experimental  data  acquired  by  having  a  human  manually  operate  a  vehicle  in  typical  outdoor 
environments  to  help  define  the  correct  logic  and  quantities  to  observe.  We  are  confident  that  if  a 
human  operator  can  navigate  the  robot  in  the  prescribed  terrain  with  only  the  information  available  in 
the  FLC  system,  then  we  can  teach  the  system  to  do  the  same.  These  methods  were  used  to  develop 
algorithms  for  the  DARPA  vehicle  shown  in  Figure  3,  which  will  also  be  used  on  the  tethered  crawler 
shown  in  Figure  2. 


Figure  1.  University  of  Idaho’s  Autonomous  Test  Vehicles 
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Figure  2.  Tethered  Crawler 


Figure  3.  DARPA  Vehicle 


Recent  events  have  shown  the  value  of  using  highly  automated  technology  in  the  battlefield. 
Autonomous  or  remote  controlled  air,  terrestrial,  and  marine  vehicles  have  accomplished  important 
missions  while  reducing  risk  to  personnel.  In  the  future,  as  greater  numbers  of  autonomous  vehicles 
are  employed,  it  is  hoped  that  lower 

LONG-TERM  GOALS 

Use  LAGR  (Learning  Applied  to  Ground  Robots)  vehicle  to  test  fuzzy  logic  obstacle  avoidance 
controller  algorithms  developed  for  crawlers  in  underwater  mine  countermeasures. 

OBJECTIVES 

A  number  of  short-term  objectives  have  been  identified  to  support  the  long-term  goal  stated  above. 
They  include: 

•  Develop  model  of  LAGR  vehicle  in  ALWSE-MC  (Autonomous  Littoral  Warfare  System 
Evaluator-Monte  Carlo) 

•  Participate  in  competitions  and  learn  from  other  competitors 
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APPROACH 

The  plan  was  to  implement  control  algorithms  from  previous  University  of  Idaho  projects  and  adapt  an 
ONR  software  package  to  test  and  optimize  the  control  algorithms  for  the  LAGR  vehicle.  The  existing 
algorithms  are  for  an  autonomous  log  skidder  (ASV30)  [1],  underwater  crawler  [2],  and  vision 
systems  [3].  The  autonomous  log  skidder  uses  human  embedding  to  train  a  fuzzy  logic  controller  to 
follow  the  same  path  we  expect  a  human  would.  The  underwater  crawler  has  a  hierarchical  fuzzy  logic 
control  system.  In  which,  a  supervisor  module  takes  inputs  from  a  path  finding  and  obstacle  avoidance 
module  to  determine  the  heading.  The  vision  system  work  uses  a  fuzzy  logic  system  that  determines  a 
path  segment’s  membership  in  slope  and  roughness  fuzzy  sets  to  determine  its  traversability.  The 
crawler  obstacle  avoidance  has  only  been  used  in  simulations  and  the  LAGR  vehicle  was  used  to  test  it 
further. 

The  ONR  software  package  ALWSE-MC  was  modified  to  simulate  the  LAGR  vehicle  and  the 
environment.  ALWSE-MC  is  a  kinematic,  statistical  AUV  mission  simulator  that  uses  point  masses  to 
simulate  the  vehicles.  To  accurately  model  the  LAGR  vehicle  and  complex  3D  environment,  changes 
were  made  to  the  simulation.  The  LAGR  vehicle  was  modeled  by  defining  the  vehicle  body  and 
sensors  (see  Figure  6).  Next  we  modified  the  environment  so  the  vehicle  can  go  through  some 
obstacles  (like  grass  and  branches)  but  stopped  by  others  (like  trunks  and  rocks). 

The  existing  control  algorithms  were  first  implemented  in  the  simulation  to  adjust  the  parameters  for 
the  LAGR  vehicle  and  check  for  problems  in  transferring  the  algorithms  across  platforms.  Then  the 
algorithms  were  transferred  to  the  University  of  Idaho  LAGR  vehicle  and  further  refined  for  navigating 
complex  environments.  The  algorithms  were  installed  and  run  on  a  similar  LAGR  vehicle  at  the 
DARPA  facilities  in  Figure  4.  These  tests  at  the  DARPA  facilities  are  monthly  competitions  among 
the  teams  in  the  project.  Since  we  are  joining  this  project  late,  our  official  testing  has  been  minimal 
and  our  testing  has  mainly  consists  of  trial  tests  on  the  UI  campus. 


Figure  4.  LAGR  Vehicle  at  DARPA  Test  Facility 
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WORK  COMPLETED 


1.  Planning 
la.  Architecture 

Global  Path  Planning  Module 

Our  control  system  was  based  on  the  premise  that  an  approximate  end-point  and  suggested  route  has 
been  defined.  The  default  path  will  be  a  straight  line  from  the  vehicle’s  start  location  to  the  desired 
destinatioa  The  path  definition  consists  of  a  series  of  points  that  connect  the  beginning  and  end  of  the 
path.  These  points  are  not  waypoints  the  vehicle  has  to  pass  through  but  rather  points  which  define  a 
route  based  on  the  topology  and  traversability  of  the  terrain.  The  main  purpose  of  the  Path  Planning 
Module  is  that  paths  may  lead  toward  the  destination  and  avoid  local  minimums  in  the  path  solution. 
The  Path  Planning  Module  also  memorizes  terrain  data  from  run  to  run  such  that  performance 
increases  with  experience. 

Local  Trajectory  Planning  Modules 

A  hierarchical  fuzzy  logic  control  system  was  developed  to  mimic  human  planning  decisions  based  on 
the  available  sensory  data  and  traversability  map  (see  Figure  5).  We  selected  specific  functions  or 
control  strategies  for  each  low  level  module.  We  then  established  the  module’s  control  structure 
including  the  input  and  output  variables  and  the  rule  base.  We  parameterized  the  membership 
functions  of  both  the  input  and  output  variables  and  selected  a  performance  index  for  the  module.  We 
used  computer  simulations  of  the  vehicle  in  the  environment  to  optimize  the  membership  functions’ 
parameters  relative  to  the  performance  index.  All  low  level  modules  provide  a  heading  (//,  quality 
factor  Q,  and  speed  recommendation.  The  Supervisor  Module  uses  quality  factors  to  arbitrate  the 
recommendations  of  the  low  level  modules  to  obtain  a  single  control  recommendation  that  the  robot 
will  use  to  navigate. 


Figure  5.  Schematic  of  Hierarchical  Fuzzy  Logic  Control  Structure 
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The  Dead  Reckoning  Module  uses  the  Local  and  Global  Pose  (Position  Estimation)  to  locate  the 
vehicle  and  determine  a  control  recommendation  that  attempts  to  keep  the  vehicle  close  to  the 
prescribed  path.  The  Trail  Following  Module  uses  a  vision  system  to  locate  openings  and  avoid 
obstacles  so  as  to  move  through  the  terrain  in  the  easiest  possible  fashion.  In  many  instances,  the  best 
route  may  not  be  along  the  prescribed  path  or  even  back  towards  the  path.  In  general,  the  vehicle  tries 
to  follow  the  easiest  path,  consistent  with  the  path  derived  by  the  Global  Planner. 

The  Hazard  Avoidance  Module  uses  classical  obstacle  avoidance  techniques  [4],  INS,  GPS  and  the 
vision  system  to  detect  dangerous  terrain  and  obstacles.  The  INS  is  used  to  prevent  the  vehicle  from 
rolling  over  or  operating  on  dangerous  slopes.  The  Hazard  Avoidance  Module  provides  a  speed  and 
heading  goal  to  the  Supervisor  Module.  The  Positional  Error  Correction  Module  is  a  fuzzy  inference 
system  that  uses  image  input  to  check  for  optical  flow  and  relate  this  to  perceived  motion  from  the  INS 
system  and  the  motor  current  to  check  and  correct  for  wheel  slip.  In  addition,  a  major  focus  on  planner 
improvement  was  in  correcting  for  GPS  drifts  and  jumps.  With  less  global  positioning  error  the 
planner  was  able  to  make  more  accurate  decisions  with  its  traversability  cost  map  in  all  areas  of  the 
controller  from  path  planning  to  trajectory  decisions. 

The  remaining  two  modules  augment  the  system  to  correct  negative  behaviors  associated  with  the  base 
system.  The  Open  Space  Module  is  designed  to  make  the  robot  steer  towards  more  open  areas  and 
thus  centers  itself  on  a  road  or  trail.  This  is  to  correct  for  the  waypoint  paths  tendency  to  closely  follow 
sets  of  obstacles  in  minimizing  a  path  distance  to  the  goal  The  Stabilizer  module  is  in  place  to  correct 
for  oscillations  in  the  Obstacle  Avoidance  Module.  In  certain  circumstances,  a  single  object  will  be  in 
the  direct  heading  of  the  vehicle.  In  these  cases,  the  Obstacle  Avoidance  Module  can  oscillate  between 
a  right  and  left  path  around  the  obstacle.  The  stabilizer  makes  the  previous  heading  decision  more 
desirable  and  thus  limits  the  oscillations. 

lb.  Simulation  and  Testing 

We  use  ALWSE-MC  to  model  the  LAGR  vehicle  for  testing  and  optimize  our  obstacle  avoidance  and 
path  planning  algorithms  before  they  are  implemented  on  the  vehicle.  Since  ALWSE  does  not  handle 
vehicle  dynamics,  a  more  specific  model  of  the  vehicle  has  been  developed.  ALWSE-MC  models 
vehicles  as  point  masses;  which  is  not  appropriate  for  modeling  the  obstacle  fields  that  the  LAGR 
vehicle  navigates.  The  vehicle  is  modeled  as  the  70cm  x  50cm  rectangle  in  Figure  6,  and  vehicle  center 
is  the  position  reported  by  ALWSE-MC.  LAGR  vehicle  sensors  are  added  because  ALWSE-MC  is 
designed  for  underwater  vehicles  and  there  are  no  suitable  camera  sensors  built  into  the  software 
package.  The  Bumble  Bee  stereo  system  is  modeled  by  returning  the  closest  obstacle  within  range 
(marked  as  X’s  in  Figure  6)  for  each  direction.  With  this  model,  the  LAGR  vehicle  can  only  see  the 
edge  of  an  obstacle  and  nothing  past  the  first  obstacle.  There  is  also  a  bumper  that  stops  the  vehicle 
during  collisions. 

The  vehicles  are  supposed  to  learn  the  terrain,  so  that  subsequent  runs  get  better  results.  The  vehicle 
divides  the  area  into  20  cm  cells  based  on  the  limited  accuracy  of  the  stereo  vision  data.  When  the 
stereo  camera  detects  an  obstacle,  the  vehicle  marks  that  cell  as  impassible,  shown  with  red  boxes  in 
Figure  6.  The  vehicle  then  uses  the  marked  cell  for  obstacle  avoidance.  Figure  7  shows  a  sample 
output  from  the  simulations  performed  on  ALWSE-MC. 
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Figure  6.  Vehicle  Model:  The  vehicle  is  modeled  as  a  70cm  x  50xm  rectangle.  There  is 
a  bumper  on  the  front  of  the  vehicle  that  stops  the  vehicle  if  anything  hits  it.  The  stereo 
system  returns  the  closest  obstacle  in  each  direction  in  the  scan  area.  Once  the  stereo 
system  picks  up  an  object  it  marks  that  cell  as  impassible  in  the  map. 


Figure  7.  Example  Simulation:  Left  Image:  planner  system  running  with  ALWSE-MC. 

Right  Image:  3D  simulator  display. 
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2.  Perception 


Fuzzy  Terrain  Classifier  Module  uses  stereo  data  from  raw  imagery  to  create  two  dimensional 
Cartesian  traversability  grids  (see  Figure  8).  The  current  system  uses  software  libraries  available 
onboard  the  robot  to  receive  and  process  image  pixels  into  three  dimensional  arrays  of  Cartesian  points 
that  can  then  be  placed  into  two  dimensional  coordinates  with  features.  The  features  of  these  grid  cells 
are  used  as  inputs  to  a  fuzzy  inference  classifier  that  rates  the  cells  traversability  and  outputs  a  ‘likely’ 
class  with  associated  quality  factor. 

The  linguistic,  fuzzy  rule  base  of  the  classifier  is  derived  through  human  embedding.  Data  from 
feature  cells  are  considered  and  rules  are  defined  by  mimicking  a  human's  choices.  The  problem  with 
real  navigation  is  that  the  environment  is  not  always  going  to  have  the  same  types  of  obstacles  and  the 
terrain  will  vary  in  color  and  texture.  For  this  reason  the  classifier  is  trained  with  sets  of  data  to 
optimize  it  for  the  current  terrain  that  it  is  navigating.  This  can  be  done  off-line  with  log  data  or  in 
real-time  as  the  vehicle  is  moving  through  the  course.  The  training  data  is  derived  from  robot 
experiences.  For  example  if  the  robot  has  successfully  traversed  a  type  of  cell,  then  the  features  within 
that  cell  can  be  considered  low  cost  and  traversable.  In  addition  dangerous  cells  can  be  marked  when 
contact  is  made  or  when  the  robot  becomes  stuck.  Besides  generating  a  cost  function  for  cells  along  the 
terrain  the  classifier  also  tries  to  associate  the  cell  with  a  particular  class  of  object  such  as  “grass”  or 
“trail”  or  “tree.”  This  information  is  used  in  the  Trail  Finding  Module  in  the  case  where  a  trail  type 
class  is  found  such  that  the  robot  will  make  an  effort  to  follow  this  trail  towards  the  goal. 

A  problem  with  using  traditional  stereo  machine  vision  techniques  is  its  short  range  limitations.  A 
goal  of  many  LAGR  teams  and  UI  is  to  develop  long  range  vision  capabilities  for  use  with  terrain 
classification.  A  method  of  doing  this  is  to  use  color  in  the  image  to  classify  pixels  that  are  out  of 
stereo  range.  This  is  usually  done  along  the  horizon  line  in  the  image.  Experiments  have  been 
conducted  on  using  pixels  that  are  outside  of  the  stereo  range  as  points  to  be  classified  and  used  in  the 
traversability  cost  map  at  further  ranges.  A  next  step  in  the  development  of  a  more  advanced  fuzzy 
inference  is  to  use  more  complex  image  processing  techniques  to  find  edges  and  lines  in  the  image 
space. 
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Figure  8.  Fuzzy  Terrain  Classifier  Module 


RESULTS 

In  simulation  the  vehicle  is  able  to  get  around  solid  walls  and  rocks  without  collisions.  The  vehicle  is 
also  able  to  determine  what  gaps  are  passable,  like  between  the  rocks  in  Figure  9.  It  can  get  through 
the  first  set,  but  it  has  to  go  around  the  second  set.  During  the  run,  the  stereo  camera  is  detecting 
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obstacles  and  putting  them  in  the  map  shown  in  Figure  10.  The  vehicle  then  uses  the  marked  cells  to 
navigate  to  the  goal. 

A  significant  amount  of  work  has  gone  into  image  processing  of  camera  sensor  data  to  accurately 
model  the  terrains  traversability.  This  system  can  be  Tun  offline  with  log  data  or  online  with  live 
camera  images.  The  system  has  already  proved  to  be  reliable  in  varying  terrain  types  and  some 
examples  of  this  are  shown  in  Figure  1 1 . 


Figure  9.  LAGR  Run:  In  this  run  the  LAGR  vehicle  navigates  its  way  around  walls  and 
rocks.  It  is  able  to  determine  which  gaps  it  can  make  it  through  and  which  ones  it 

cannot. 


Figure  10.  LAGR  Map:  The  map  that  the  LAGR  vehicle  developed  during  the  run  shown 
in  LAGR  Run  of  Figure  8.  The  marked  cells  are  all  that  the  LAGR  vehicle  has  for 

navigation. 


n 


Figure  1 1 .  Terrain  Perception:  The  right  image  shows  raw  color  data.  The  middle  figure 
is  the  same  image  with  masking  to  show  areas  of  obstacles  as  classified  with  the  fuzzy 
logic  traversability  classifier.  The  right  image  is  the  data  projected  into  a  2  dimensional 
Cartesian  grid  that  uses  magenta  to  mark  dangerous  obstacles. 

IMPACT/APPLICATIONS 

There  is  no  other  group  we  know  of  that  are  currently  not  using  the  LAGR  baseline  system  and 
therefore  few  teams  are  using  simulation  to  test  and  optimize  their  planner  algorithms  before  uploading 
them  for  real  time  testing.  We  feel  that  adapting  ALWSE-MC  to  simulate  the  LAGR  vehicle 
contributes  to  the  LAGR  program  by  allowing  for  quicker  iterations  and  tuning  of  the  control 
algorithms. 

By  putting  our  existing  algorithms  into  the  LAGR  vehicle  we  can  further  develop  the  log  skidder 
control  algorithms  and  test  the  crawler  control  algorithms  on  a  real  vehicle.  The  Hierarchal  control 
systems  developed  for  the  LAGR  vehicle  are  designed  to  be  modular  and  general  enough  to  be 
imported  to  various  autonomous  vehicles  such  that  tuning  of  parameters  is  minimized  and  performance 
is  optimized.  The  human  embedding  method  creates  a  non-linear,  complex  controller  that  closely 
matches  an  expert  driver’s  actions.  This  is  of  direct  interest  to  ONR’s  mine  counter  measure  programs 
where  heterogeneous  underwater  crawlers  will  be  able  to  use  similar  software  to  successfully  navigate 
rocky  terrain  in  an  efficient  manner. 

In  addition  to  the  control  systems  developed,  the  learning  algorithms  used  to  automatically  train  the 
vehicle  allow  the  systems  to  be  more  robust  and  adapt  to  unknown  terrain  types.  This  online  learning 
is  crucial  to  successful  missions  in  dynamic  underwater  environments  where  communications  and 
endurance  are  limited. 
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